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The  APEX  method  and  real-time  blind  deconvolution  of 
scanning  electron  microscope  imagery 

Alfred  S.  Carasso* *  David  S.  Bright^  Andras  E.  VladaB 


Abstract 

Loss  of  resolution  due  to  image  blurring  is  a major  concern  in  electron  microscopy.  The 
point  spread  function  describing  that  blur  is  generally  unknown.  This  paper  discusses  the  use 
of  a recently  developed  FFT-based  direct  (non-iterative)  blind  deconvolution  procedure,  the 
APEX  method,  that  can  process  512  x 512  images  in  less  than  a minute  on  current  desktop 
platforms.  The  method  is  predicated  on  a restricted  but  significant  class  of  shift- invariant 
blurs,  consisting  of  finite  convolution  products  of  Levy  probability  density  functions.  Such 
blurs  considerably  generalize  Gaussian  and  Lorentzian  point  spread  functions.  In  this  paper, 
the  method  is  applied  to  a variety  of  original  SEM  micrographs,  and  shown  to  be  useful  in 
enhancing  and  detecting  fine  detail  not  otherwise  discernible.  Quantitative  sharpness  analysis 
of  'ideal  sample’  micrographs,  shows  that  APEX  processing  can  actually  produce  sharper  im- 
agery than  is  achievable  with  optimal  microscope  settings. 

Subject  terms:  electron  microscopy,  real-time,  image  deblurring,  blind  deconvolution,  Levy 
density  functions,  APEX  method,  SECB  method,  SEM  images. 


1.  Introduction 

Loss  of  resolution  due  to  image  blurring  is  a major  concern  in  scanning  electron  microscopy  (SEM). 
Moreover,  unless  specifically  measured,1,2  the  shape  of  the  electron  beam  is  not  known  to  the  mi- 
croscopist.  Hence,  the  point  spread  function  (psf)  describing  the  blur  is  generally  unknown.  This 
paper  discusses  the  use  in  electron  microscopy  of  a recently  developed  blind  deconvolution  proce- 
dure, the  APEX  method,*'4  which  sharpens  the  image  while  simultaneously  increasing  contrast  and 
brightness.  The  degree  of  enhancement  can  be  adjusted  by  appropriate  choice  of  input  parame- 
ters. To  the  extent  permitted  by  the  level  of  data  noise,  the  APEX  method  sharpens  the  image 
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by  restoring  some  of  the  high  frequency  content  that  had  been  attenuated  in  the  course  of  imaging 
the  sample.  In  this  paper,  the  method  is  applied  to  a variety  of  original  SEM  micrographs  and 
shown  to  be  useful  in  enhancing  and  detecting  hue  detail  not  otherwise  discernible.  In  addition, 
quantitative  sharpness  analysis  of  ‘ideal  sample  micrographs,2  shows  that  APEX  processing  can 
actually  produce  sharper  imagery  than  is  achievable  with  optimal  microscope  settings. 

As  in  all  inverse  problems,  successful  use  of  the  APEX  method  requires  a-priori  knowledge  about 
the  solution.  Here,  such  prior  knowledge  takes  the  form  of  training  and  experience  on  the  part  of 
the  microscopist,  whose  judgment  is  called  upon  to  distinguish  genuine  features  in  the  presence  of 
noise  and  visually  select  the  ‘optimal’  reconstruction.  The  images  we  are  concerned  with  come  from 
scanning  electron  beam  instruments  such  as  the  field  emission  gun  scanning  electron  microscope 
(FEGSEM),  a high  resolution  instrument,  and  the  environmental  scanning  electron  microscope 
(ESEM),  a lower  resolution  instrument  with  more  flexible  sample  handling  capability.  In  a future 
report,  we  shall  explore  the  possible  use  of  APEX  methodology  to  produce  a quantitative  measure 
of  SEM  imaging  performance. 

Blind  deconvolution  seeks  to  deblur  an  image  without  knowing  the  cause  of  the  blur.  This 
is  a difficult  mathematical  problem  in  which  ill-conditioning  is  compounded  with  non-uniqueness 
of  solutions.  A priori  constraints  reduce,  but  do  not  entirely  eliminate,  the  multiplicity  of  solu- 
tions. While  many  of  these  solutions  are  physically  meaningless,  there  are  in  general  several  useful 
solutions.4  Most  approaches  to  blind  deconvolution  are  iterative  in  nature,  and  aim  at  simultaneous 
reconstruction  of  both  the  psf  and  the  deblurred  image.  However,  that  iterative  process  may  be- 
come ill-behaved  and  develop  stagnation  points  or  diverge  altogether. 5 When  the  iterative  process 
is  stable,  several  thousand  iterations  may  Ire  necessary  to  resolve  fine  detail.  In  general,  iterative 
algorithms  are  not  well-suited  for  real-time  processing  of  large  size  images  of  complex  objects. 

The  APEX  method  is  an  FFT-based  direct  (non-iterative)  blind  deconvolution  technique  that 
can  be  used  in  real-time  applications.  It  was  developed  and  analyzed  in  Ref.  3,  and  documented 
there  with  numerous  applications  to  synthetically  blurred  images.  More  recently,4  the  method  was 
successfully  applied  to  a variety  of  real  blurred  images  obtained  from  diverse  imaging  modalities, 
including  astronomical,  aerial,  and  Landsat  images,  MR  I and  PET  brain  scans,  as  well  as  other 
types  of  interesting  images.  However,  not  all  images  can  be  usefully  enhanced  with  the  APEX 
method. 

Rather  than  considering  the  blind  deconvolution  problem  in  full  generality,  the  APEX  method  is 
predicated  on  a restricted  but  significant  class  of  shift-invariant  blurs,  the  class  G.  ’ ' which  consists 
of  finite  convolution  products  of  2-D  radially  symmetric  Levy  ‘stable’  probability  density  functions.8 
That  class  considerably  generalizes  Gaussian  and  Lorentzian  psfs.  The  motivation  for  using  the  class 
G as  the  framework  for  the  APEX  method,  is  that  numerous  electron-optical  imaging  devices  have 
psfs  in  class  G.  or  have  psfs  that  can  be  well-approximated  by  class  G psfs.  This  is  documented  in 
section  2.  Apparently,  as  will  be  shown  in  sections  7 and  8 below,  the  class  G can  also  Ire  usefully 
applied  to  electron  microscope  imagery. 

The  APEX  method  is  based  on  detecting  the  signature  of  a class  G psf  from  1-D  Fourier 
analysis  of  the  blurred  image.  That  detected  psf  is  then  used  in  a separate  FFT-based  direct  image 
deblurring  procedure,  the  SECB  method.1  ' to  produce  the  deblurred  image.*  When  the  APEX 
method  is  useful,  blind  deconvolution  of  512  x 512  images  can  be  accomplished  in  less  than  a minute 


* United  States  patents  have  been  issued  on  parts  of  the  work  described  in  Refs.  3-7. 
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Figure  1:  Comparing  APEX  processing  with  unsharp  masking  on  a synthetically  blurred  image. 

(A)  Original  8-bit  512  x 512  MRI  sagittal  brain  image.  (B)  Synthetically  blurred  MRI  image  stored 
in  8-bit  precision.  (C)  Sharpening  of  image  (B)  using  unsharp  masking.  (D)  Sharpening  of  image 

(B)  using  APEX  method. 
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on  current  desktop  platforms.  As  illustrated  in  Figure  1,  APEX  processing  can  produce  significantly 
sharper  images  than  is  generally  possible  with  unsharp  masking. 

An  important  aspect  of  blind  deconvolution  can  be  illustrated  by  means  of  the  following  analogy. 
Imagine  several  experienced  photographers  located  at  approximately  the  same  vantage  point,  and 
simultaneously  photographing  an  identical  scene.  In  general,  different  images  will  be  produced 
through  use  of  different  cameras,  film,  light  filters,  exposures,  printing,  and  the  like.  While  each 
image  is  a correct  visual  representation  of  the  original  scene,  the  images  will  differ  from  one  another 
in  contrast,  brightness,  sharpness,  and  numerous  other  details.  A pixel  by  pixel  comparison  of 
these  images  would  reveal  substantial  differences.  Which  of  the  several  photographs  is  the  true 
version  of  reality  cannot  easily  be  answered.  They  are  all  useful  approximations.  An  analogous 
phenomenon  occurs  in  blind  deconvolution.  As  illustrated  in  section  4,  given  a blurred  image, 
there  are  in  general  many  useful  reconstructions  that  are  possible.  These  reconstructions  may  differ 
substantially  from  one  another  at  individual  pixels,  while  being  correct  visual  representations  of  the 
object  that  was  imaged.  This  is  an  inherent  non-uniqueness  property  of  the  blind  deconvolution 
problem,  independently  of  any  particular  algorithm  that  might  be  used  to  solve  that  problem.4 

A basic  property  of  the  APEX  method  is  that  it  generally  provides  several  psfs  that  can  be  used  to 
obtain  useful  reconstructions  of  the  same  blurred  image.  As  in  the  case  above,  these  reconstructions 
differ  from  one  another  at.  individual  pixels  while  being  visually  correct.  As  already  noted,  a-priori 
knowledge  about  the  desired  solution  is  a necessary  ingredient  for  solving  ill-posed  inverse  problems. 
Such  knowledge  is  expected  to  guide  the  user  in  his  selection  of  the  best  reconstruction.  Whether 
or  not  APEX  processing  is  beneficial  in  any  given  case  can  usually  be  quickly  decided.  For  images 
where  APEX  processing  provides  useful  enhancement,  hue  tuning  of  parameters  enables  the  user 
to  adjust  the  quality  of  the  reconstruction,  within  the  limitations  imposed  by  the  level  of  noise  in 
the  blurred  image. 

2.  Imaging  systems,  Levy  point  spread  functions,  and  the  class  G 

Point,  spread  functions  h,(x,  y)  can  be  viewed  as  2-D  probability  density  functions  since  they  are  non- 
negative  and  integrate  to  unity.  The  Fourier  transform  //.(£,//)  °f  a psf  h(x,y)  is  called  the  optical 
transfer  function  (otf).  Knowledge  of  the  otf  determines  the  psf  and  vice  versa.  Note  that  while  the 
psf  h(x,y)  is  always  non-negative,  the  otf  h(£,rj)  is  complex-valued  in  general.  The  absolute  value 
of  the  otf  is  called  the  modulation  transfer  function  (mtf). 

Gaussian  psfs  are  ubiquitous  in  imaging  systems  but  represent  only  one  example  of  the  general 
class  of  Levy  stable  densities.  In  the  2-D  radially  symmetric  case,  Levy  stable  densities  h(x,  y)  can 
be  defined  implicitly  in  terms  of  their  Fourier  transforms  by 

h{Z,ri)=(  h{x:y)e-2^x+r)y)dxdy  = e-a{e+r,2)\  a > 0.  0 < (3  < 1.  (1) 

Jr2 

For  general  (3,  h(x,y)  in  (1)  is  not  known  in  closed  form.  However,  the  cases  (3=1  and  (3  = 1/2 
correspond  to  Gaussian  and  Lorent.zian  (or  Cauchy)  densities  respectively.  When  (3=1,  h{x,  y) 
has  slim  tails  and  finite  variance.  For  0 < (3  < 1,  h(x,y)  has  fat  tails  and  infinite  variance.  The 
occurrence  and  analysis  of  Levy  processes  in  the  physical  sciences  are  subjects  of  significant,  current 
interest.9-12 
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Image  intensifiers,  CCDs,  and  numerous  other  electron-optical  devices  are  used  in  a wide  variety 
of  astronomical,  industrial,  biomedical,  military,  and  surveillance  imaging  systems.  A systematic 
study  of  electron-optic  mtf  measurements  has  led  to  the  important  empirical  discovery,13-1''  that 
an  extensive  variety  of  electronic  imaging  devices  have  otfs  h(C7l)  that  are  well-described  by  (1) 
with  1/2  < (3  < 1.  In  particular,  non-Gaussian  behavior  is  often  found  in  electron-optic  imaging 
systems.  For  any  given  device,  the  values  of  a and  (3  can  be  determined  using  specialized  graph 
paper.  The  characterization  (1)  is  useful  in  other  areas  of  optics.  The  diffraction-limited  otf  for  a 
perfect  lens  [16,  p.  154],  can  be  approximated  over  a wide  frequency  range  by  (1),  with  / 3 = 3/4 
and  a a function  of  the  cutoff  frequency.1'  The  otf  for  long-exposure  imaging  through  atmospheric 
turbulence,18  is  known  to  be  given  by  (1)  with  / 3 = 5/6,  and  a determined  by  atmospheric  conditions. 
In  Ref.  19,  mtf  data  for  56  different  kinds  of  photographic  him  are  analyzed.  Good  agreement  is 
found  when  these  data  are  fitted  with  (1),  and  the  pairs  {a,f3)  characterizing  each  of  these  56  mtfs 
are  identified.  It  is  found  that  36  types  of  him  have  mtfs  where  1/2  < (3  < 1.  The  remaining  20 
types  have  mtfs  with  values  of  (3  in  the  range  0.265  < (3  < 0.475. 

For  cascaded  imaging  systems  composed  of  several  elements  satisfying  (1),  the  resulting  lumped 
otf  has  the  form 

/?.(£,  rj)  = e~  MC+ri2 A _ a.  > o5  0 < (3%  < 1.  (2) 

Such  an  expression  can  also  be  used  to  best-ht  a large  variety  of  empirically  obtained  optical  transfer 
functions,  by  varying  the  parameters  at,  [37 , and  J.  We  define  G to  be  the  class  of  all  point  spread 
functions  h(x,y)  satisfying  (2).  Note  that  class  G psfs  have  non-negative  Fourier  transforms.  This 
is  not  true  of  psfs  in  general.  For  example,  the  optical  transfer  function  for  uniform  optical  defocus 
blur  is  the  ‘sombrero  function'  [20,  p.  72],  which  develops  negative  oscillations. 

Motivated  by  these  considerations,  we  consider  image  deblurring  problems  with  psfs  in  G.  In 
the  absence  of  noise,  we  have 

Hfe  = / h(x  - u , y - v ) fe ( u , v)dudv  = h(x,  y)  <g>  fe(x,  y)  = ge(x , y),  (3) 

where  ge{x,  y)  is  the  blurred  image  that  would  have  been  recorded  in  the  absence  of  noise,  fe(x,  y)  is 
the  exact  unblurred  image,  h(xyy)  is  a point  spread  function  in  class  G,  and  denotes  convolution. 
In  general,  the  given  blurred  image  g(x,y)  includes  noise,  which  is  viewed  as  a separate  additional 
degradation, 

g{x,  y ) = ge{x , y)  + n(.x,  y ).  (4) 

Here,  n(x,  y)  represents  the  cumulative  effects  of  all  errors  affecting  final  acquisition  of  the  digitized 
array  g(x:y).  This  includes  multiplicative  noise,  where  n(x,y)  may  be  a nonlinear  function  of 
fe(x,y).  Neither  ge(xyy)  nor  n(x,y)  are  actually  known,  only  their  sum  g(xyy).  Hence,  rather  than 
(3),  we  must  consider  the  more  difficult  problem 

Hf  = h(x,  y)  <S>  f{x,  y)  = g{x , y).  (5) 

As  is  well-known,22  even  though  n(x,  y)  may  be  presumed  small,  its  presence  in  (4)  has  a profound 
impact  on  the  solution  of  the  ill-posed  equation  (5).  A survey  of  the  best-known  linear  and  nonlinear 
algorithms  for  handling  (5)  may  be  found  in  Ref.  7.  The  strategy  is  to  find  an  approximate  solution 
P{x,  y)  such  that  h(x,  y)  <g>  P(x,y)  ~ g{x,  y)  and  such  that  ||  p — fe  ||  is  small.  For  psfs  in  class 
G,  the  SECB  method  outlined  in  section  3 is  particularly  effective. 
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2.1.  Connection  with  SEM  imaging 

In  interpreting  (3)  in  relation  to  SEM  imaging,  a conceptual  framework  that  lias  been  used  in 
several  recent  studies23  is  helpful.  Let  s(x,y ) be  a function  describing  the  actual  sample.  The  SEM 
converts  s(x,y ) into  an  image  i(x,y ),  where 


i{x,y)  = I[s{x,y)].  (6) 

Here,  / is  the  instrument  transform  and  is  partly  nonlinear.  The  nonlinear  component  of  /,  call 
it  M,  consists  of  the  details  of  the  nonlinear  interaction  between  the  electrons  and  the  material. 
This  component  can  be  studied23,24  by  Monte  Carlo  simulations  applied  to  electron  trajectories, 
but  is  not  readily  invertible.  The  other  component  of  I,  call  it  q,  describes  blurring  due  to  the 
electron  beam  point  spread,  along  with  some  of  the  instrument’s  electronics.  That  component  is 
often  represented  as  a convolution.  Therefore,  in  the  absence  of  noise, 

i{x,  V)  = q{x,  y)  ® M[s{x,  y)].  (7) 

Comparing  (7)  with  (3),  we  are  led  to  identify  i(x,y)  with  ge(x,y ),  M[s(x,y)]  with  fe(x,y),  and 
q{x,y)  with  h(x,y).  Thus,  blind  deconvolution  of  (5)  using  the  APEX  method,  may  be  interpreted 
as  an  attempt  to  recapture  M[s(x,y)\  from  noisy  data,  by  undoing  blurring  due  primarily  to  the 
unknown  electron  beam  point  spread. 


3.  Deblurring  with  the  SECB  method 


The  SECB  method  is  a direct  F FT- based  image  deblurring  technique  designed  for  equations  of  the 
form  (5)  when  h,(x,y)  is  known  and  belongs  to  G.  A complete  discussion  of  that  method,  together 
with  error  bounds  and  comparisons  with  other  methods,  may  be  found  in  Refs.  5-7.  Significantly, 
the  SECB  method  does  not  impose  smoothness  requirements,  such  as  a-priori  bounds  on  the  Lapla- 
cian  or  other  derivatives  of  the  unknown  image  f(x,  y}.  This  is  an  important  consideration  since 
many  images  have  sharp  edges  and  other  localized  lion-differentiable  features. 

For  class  G psfs,  we  may  define  fractional  powers  Hl , 0 < / < 1,  of  the  convolution  integral 
operator  H in  (5)  as  follows 


,,)/(£,,,)},  o < t < i. 


(8) 


Class  G psfs  are  intimately  related  to  diffusion  processes,  and  solving  (5)  is  equivalent  to  finding 
the  initial  value  u(x,y,0)  — f{x,y)  in  the  backwards  in  time  problem  for  the  generalized  diffusion 
equation 


U,  = - Eil  Ai(  — A f'K  A.  = Q,:(47r2)-ft, 


0 < t < 1. 


u(x,y,  i)  = g{x,y). 


(9) 


When  this  initial  value  f(x,  y)  is  known,  u{x,  y,  t)  = H1  f is  the  solution  of  (9)  at  time  t.  The  SECB 
method  is  a regularization  method  for  solving  the  ill-posed  problem  (9)  that  takes  into  account  the 
presence  of  noise  in  the  blurred  image  data  g(x,  y)  at  t = 1.  The  SECB  deblurred  image  p{x,  y)  is 
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an  approximation  to  fe(x,y)  that  is  obtained  in  closed  form  in  Fourier  space.  With  z denoting  the 
complex  conjugate  of  2, 


fHC  n)  = 


h{Cn)9(tv) 

??)|2  + K~2\I  — hs(^:r])\2 


(10) 


leading  to  /I  (x,  y)  upon  inverse  transforming.  Here,  the  regularization  parameters  A , s are  positive 
constants  that  are  chosen  based  on  a-priori  information.  ’ ' I11  blind  deconvolution  applications  of 
the  SECB  method,  the  APEX-detected  parameters  cq,  fa  are  used  in  (2)  which  is  then  input  into 
(10).  In  practice,  FFT  algorithms  are  used  to  obtain  p(x,y).  This  may  result  in  individual  pixel 
values  that  are  negative.  Accordingly,  all  negative  values  are  reset  to  the  value  zero.  For  512  x 512 
images,  a single  trial  SECB  restoration  requires  about  one  second  of  cpu  time  on  current  desktop 
workstations.  We  may  also  form  and  display 


T (•'••//•/)  = H'f(x,y), 


(ID 


for  selected  decreasing  values  of  / lying  between  1 and  0.  This  simulates  marching  backwards  in  time 
in  (9),  and  allows  monitoring  the  gradual  deblurring  of  the  image.  As  t — > 0 the  partial  restorations 
C(x ,y,t)  become  sharper.  However,  noise  and  other  artifacts  typically  become  more  noticeable 
as  t — » 0.  Such  slow  motion  deconvolution  allows  detection  of  features  in  the  image  before  they 
become  obscured  by  noise  or  ringing  artifacts.  As  will  be  seen  below,  marching  backwards  in  time 
is  an  important  element  in  the  APEX  method. 


4.  Non-uniqueness  in  blind  deconvolution 

Blind  deconvolution  of  images  is  a mathematical  problem  that  is  not  fully  understood.  Well- 
documented  examples  of  the  kinds  of  behavior  that  may  occur  are  of  particular  interest.  In  this 
section,  we  highlight  important  non-uniqueness  aspects  of  that  problem  that  are  helpful  in  under- 
standing the  results  of  the  APEX  method.  Let  fe(x,y ) be  a given  exact  sharp  image,  let  h(x,y)  be 
a Levy  point  spread  function,  and  let  g(x , y)  = h(x,  y)  <S>  /e(.T,  y)  + noise.  We  shall  show  that  given 
the  blurred  image  g(x,y ),  there  are  in  general  many  point  spread  functions  hi{x,  y)  ^ h(x.  y)  that 
deblur  g{x,y ),  producing  useful  reconstructions  fi(x,y ) ^ fe(x,y),  with  hi(x , y)  0 fi(x , y)  ~ g{x,  y). 

The  sharp  512  x 512  New  Orleans  cathedral  image  fe(x,  y)  in  Figure  2(A)  was  synthetically 
blurred  by  convolution  with  a Cauchy  density  h.(x,  y)  with  cro  = 0.075,  /i0  = 0.5.  This  produced  the 
blurred  image  g(x,y)  in  Figure  2(B).  I11  this  experiment,  g{x,  y)  was  computed  and  stored  in  16-bit 
precision.  Thus,  g{x,y)  differs  from  ge(x,y)  by  the  effects  of  16-bit.  rounding  noise.  Deblurring 
that  image  with  the  correct  psf  values  a — 0.075,  (I  = 0.5,  produces  Figure  2(C).  As  expected, 
this  is  in  excellent  visual  agreement  with  fe(x,y)  in  Figure  2(A).  However,  Figure  2(D)  is  another 
useful  enhancement  of  Figure  2(B).  It  was  obtained  using  a Levy  density  with  values  ( ct,/3 ) where 
a > o-o,  /i  < (do,  and  it  differs  from  Figure  2(A)  in  contrast,  brightness,  and  sharpness  of  detail. 
Here,  a = 0.239767,  j3  = 0.385568.  Note  the  indented  ‘blind  window’  highlighted  in  the  left 
lateral  tower  in  Figure  2(D).  This  architectural  detail  is  barely  discernible  in  Figure  2(A),  and 
not  identifiable  in  Figure  2(C).  Both  deblurred  images  (C)  and  (D)  were  obtained  using  the  SECB 
method  with  s = 0.001  and  I\  = 100.  One  dimensional  cross  sections  of  the  two  distinct  psfs  used  in 
Figure  2 are  displayed  in  Figure  3.  To  facilitate  comparison,  the  two  psfs  in  Figure  3 are  normalized 
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Figure  2:  Non-uniqueness  in  blind  deconvolution.  Distinct  point  spread  functions  exist  that  produce 
useful  reconstructions  from  the  same  blurred  image.  (A)  Original  sharp  512  x 512  New  Orleans 
cathedral  image.  (B)  16-bit  synthetically  blurred  image  created  by  convolution  with  Lorentzian 
density  with  cv  = 0.075,  (3  = 0.5.  (C)  Deblurring  of  image  (B)  using  correct  parameters  o-  = 
0.075,  [3  — 0.5.  (D)  Deblurring  of  image  (B)  using  a = 0.239767,  (3  — 0.385568.  Image  (D) 
differs  from  image  (C)  in  contrast,  brightness  and  sharpness  of  detail.  In  particular,  indented  ‘blind 
window'  highlighted  in  left  lateral  tower  in  image  (D),  is  not  discernible  in  image  (C).  Deblurred 
images  obtained  using  SECB  procedure  with  s = 0.001  and  I\  = 100. 


THE  APEX  METHOD  IN  ELECTRON  MICROSCOP\ 


9 


SPATIAL  VARIABLE  X (IN  PIXELS ) 


Figure  3:  Two  distinct  point  spread  functions  that  deblur  image  (B)  in  Figure  F Curves  C and  D 
are  1-D  cross  sections  of  the  512  x 512  psfs  that  respectively  produced  images  (C)  and  (D)  in  Figure 
2.  To  facilitate  comparison,  curves  were  normalized  to  unit  maximum.  These  psfs  also  exhibit 
distinct  heavy  tail  behavior. 
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so  as  to  have  a maximum  value  of  1.0.  These  psfs  also  exhibit  distinct  heavy  tail  behavior  not 
shown  in  Figure  3.  There  are  obviously  many  other  distinct  psfs  lying  between  these  two  curves 
that  produce  useful  reconstructions.  Convolution  of  each  reconstruction  with  its  corresponding  psf 
in  Figure  3,  reproduces  the  blurred  image  in  Figure  2(B),  to  within  a small  error. 

5.  SEM  images  and  convex  Fourier  transforms 

Tin1  APEX  method  is  a blind  deconvolution  technique  based  on  detecting  class  G psf  signatures 
by  appropriate  1-D  Fourier  analysis  of  the  blurred  image  g(x,y).  The  detected  psf  parameters  are 
then  input  into  the  SECB  algorithm  to  deblur  the  image.  The  Fourier  transform  is  the  primary 
mathematical  tool  in  each  of  these  steps.  Accordingly,  the  qualitative  behavior  in  Fourier  space  of 
a large  class  of  SEM  images  is  of  interest. 

Let,  fe{x,y ) be  an  exact  sharp  image  as  in  (3).  Since  fe(x,  y)  > 0 

\.fe{t  n)\  < / fe{x,y)dxdy  = fe{ 0,0)  = a > 0.  (12) 

./ft-1 

Also,  since  ge(x,y)  = h(xgy)  <g>  fe(x,y)  and  h(x,y)  is  a probability  density, 

9e{ 0,  0)  = [ ge{x,  y)dxdy  = [ fe(x,  y)dxdy  = fe{ 0,  0)  = a > 0.  (13) 

./ft-1  ./ft2 

Using  a as  a normalizing  constant,  we  may  normalize  Fourier  transform  quantities  q(£,  ?/)  by  dividing 
by  a.  Let 

q*itd)  = U4) 

denote  the  normalized  quantity.  The  function  |/e  (£,?/) | is  highly  oscillatory,  and  0 < \ fe  | < 1. 
Since  fe(x , y)  is  real,  its  Fourier  transform  is  conjugate  symmetric.  Therefore,  the  function  |/e  (£,  /;)  | 
is  symmetric  about  the  origin  on  any  line  through  the  origin  in  the  (£,  ?/)  plane.  The  same  is  true 
for  the  blurred  image  data  |g*(£,//)|. 

All  blurred  images  in  sections  5 through  8 are  of  size  512  x 512  and  quantized  at  8-bits  per 
pixel.  For  any  blurred  image  g ( x , y ) , the  discrete  Fourier  transform  is  a 512  x 512  array  of  complex 
numbers,  which  we  again  denote  by  (/(£,? 7)  for  simplicity.  The  ‘frequencies’  £,77  are  now  integers 
lying  between  —256  and  256,  and  the  zero  frequency  is  at  the  center  of  the  transform  array.  This 
ordering  is  achieved  by  pre-multiplying  g(x,y)  by  ( — l)x+y.  We  shall  be  interested  in  the  values 
of  such  transforms  along  single  lines  through  the  origin.  The  discrete  transforms  |<7*(£,  0)|,  and 
\g* (0,  p) | are  immediately  available.  Image  rotation  may  be  used  to  obtain  discrete  transforms 
along  other  directions.  All  1-D  Fourier  domain  plots  shown  in  this  paper  are  taken  along  the  axis 
77  = 0 in  the  (£,  //)  plane.  In  these  plots,  the  zero  frequency  is  at  the  center  of  the  horizontal  axis, 
and  the  graphs  are  necessarily  symmetric  about  the  vertical  line  £ = 0.  Examples  of  such  plots  are 
shown  in  Figure  4. 

The  class  of  SEM  images  g(x,y)  considered  in  the  present  paper  can  be  described  in  terms  of 
the  behavior  of  In  \g*{C  7/)j  along  single  lines  through  the  origin  in  the  (£,7/)  plane.  While  local 
behavior  is  highly  oscillatory,  global  behavior  is  generally  monotone  decreasing  and  convex  on  £ > 0. 
This  is  shown  in  Figure  4(A)  for  a typical  SEM  image  along  the  line  //  = 0,  and  similar  behavior  is 
found  along  other  lines  through  the  origin  in  the  (£,77)  plane.  A least  squares  fit  to  the  oscillatory 
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Figure  4:  APEX  detection  of  psf  parameters  for  types  of  images  considered  in  this  paper.  (A) 
Behavior  of  logarithm  of  normalized  Fourier  transform,  In  \g*[C  0)|,  in  typical  SEM  image  g(x , y). 
While  local  behavior  is  highly  oscillatory,  global  behavior  is  generally  monotone  decreasing  and 
convex  on  £ > 0.  (B)  Least  squares  fit  to  In  \g*{C  0)|  with  u(£)  = — aj£|2/i  — 3.85,  (solid  line). 
Fit  develops  well-formed  cusp  at  £ = 0 and  returns  a = 0.5346,  f3  = 0.2097. 

trace  in  Figure  4(A)  with  a smooth  curve,  provides  a good  representation  of  this  global  monotone 
convexity  property  on  £ > 0.  (A  convex  function  is  such  that  given  any  two  distinct  points  A and 
B on  its  graph,  the  straight  line  segment  joining  A and  B lies  above  the  graph.)  Many  SEM  images 
exhibit  similar  globally  monotone  convex  behavior  in  Fourier  space.  Moreover,  such  behavior  is 
also  found  in  other  types  of  imagery,  unrelated  to  electron  microscopy.  In  Ref.  3,  a large  class 
of  images  with  that  property  was  exhibited  and  denoted  by  W.  The  SEM  images  considered  here 
may  be  loosely  characterized  as  being  in  class  W.  Not  all  blurred  images  may  be  so  characterized. 
Application  of  the  APEX  method  to  cases  where  global  behavior  in  In  \g*{C  ?/)|,  away  from  the 
origin,  is  monotone  decreasing  and  concave,  are  discussed  in  Ref  3.  Use  of  the  APEX  method  in 
the  manner  described  below  is  intended  only  for  blurred  images  whose  Fourier  space  behavior  is 
analogous  to  that  shown  in  Figure  4(A). 


6.  Marching  backwards  in  time  and  the  APEX  method 

The  APEX  method  is  based  on  the  following  observations.  In  the  basic  relation 

g{x,  y)  = h{x,  y)  <g>  fe{ x,  y)  + n{x , y ), 
we  may  safely  assume  that  the  noise  n(x,  y)  satisfies 


./  R2 


\n(x , y ) I dxdy  <C  / fe{x , y)dxdy  = a > 0, 
Jr 2 


so  that, 

|n*(£,7?)|  <C  1. 

Consider  the  case  where  the  otf  is  a pure  Levy  density  /?.(£,  •//)  = e_Q^'+7?’)/A  Since  g 

in  \?(Cn)\  = i"  + n*(C,i?)l- 


g e + n 


15) 


(16) 


T7 


(18) 
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Figure  5:  Enhancement  of  8-bit  blurred  MRI  brain  image  by  marching  backwards  from  t — 1 with 
APEX-detected  psf.  Sequence  shows  gradual  increase  in  resolution  as  t decreases.  Undesirable 
artifacts  near  t = 0 indicate  progression  backwards  in  time  has  continued  too  far.  Here,  best 
results  occur  at  values  of  t such  that  0.3  < t < 0.4  . 

Let.  Q — {(£,  ?/)  | £2  + rj2  < u2}  be  a neighborhood  of  the  origin  where 

e-a(52+’,2)',|/«*(COI»|n*(C>?)|.  (19) 

Such  an  Q exists  since  (19)  is  true  for  £ = rj  = 0 in  view  of  (17).  The  radius  to  > 0 of  Q decreases 
as  a and  n increase.  For  (£,  //)  6 0 we  have 

in  \g*(C h)|  ~ ~a{(2  + v2)1'  + In (20) 

Because  of  the  radial  symmetry  in  the  psf,  it  is  sufficient  to  consider  (20)  along  a single  line  through 
the  origin  in  the  (£,  //)  plane.  Choosing  the  line  = 0,  we  have 

lii|r(C0)|«-a|?|2,J  + ln|/e*(5,0)|.  |f|<w.  (21) 

Some  type  of  a-priori  information  about  fe{x,y ) is  necessary  for  blind  deconvolution.  In  (21), 
knowledge  of  In  |/e  (£,0)|  on  |£|  < c o would  immediately  yield  cv|^|“'  on  that  interval.  Moreover, 
any  other  line  through  the  origin  could  have  been  used  in  (20).  However,  such  detailed  knowledge 
is  unlikely  in  practice.  The  APEX  method  seeks  to  identify  a useful  psf  from  (21)  without  any  prior 
knowledge  about  In  | fe  (£,  0)|.  The  method  assumes  instead  that  fe{x,  y ) is  a recognizable  object, 
and  typically  requires  several  interactive  trials  before  locating  a suitable  psf.  As  previously  noted, 
such  trial  SECB  restorations  are  easily  obtained.  Here,  prior  information  about  fe(: r,  y)  is  disguised 
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in  the  form  of  user  recognition  or  rejection  of  the  restored  image,  and  that  constraint  is  applied  at 
the  end  of  the  reconstruction  phase,  rather  than  at  the  beginning  of  the  detection  phase. 

In  the  absence  of  information  about  In  | fe  (£,  0)|,  we  replace  it  by  a negative  constant  —.4  in 
(21).  For  any  A > 0,  the  approximation 

ln|j*(C0)l“-«K|2'J--4,  (22) 

is  not  valid  near  £ = 0,  since  the  curve  u(£)  — —a\^\2>3  — .4,  has  —.4  as  its  apex.  Choosing  a value 
of  A > 0,  we  best  fit  In  |g*(£,  0)|  with  u(£)  = —a\L,\2fi  — A on  the  interval  |£|  < cj,  using  nonlinear 
least  squares  algorithms.  The  resulting  fit,  is  close  only  for  £ away  from  the  origin.  The  returned 
values  for  a and  (3  are  then  used  in  the  SECB  deblurring  algorithm.  Different  values  of  .4  return 
different  pairs  (cq/i).  Experience  indicates  that  useful  values  of  ,4  generally  lie  in  the  interval 
2 < A < 6.  Increasing  the  value  of  .4  decreases  the  curvature  of  u{£)  at  £ = 0,  resulting  in  a 
larger  value  of  (3  together  with  a smaller  value  of  a . A value  of  A > 0 that  returns  (3  > 1 is  clearly 
too  large,  as  (3  > 1 is  impossible  for  probability  density  functions.8  Decreasing  .4  has  the  opposite 
effect,  producing  lower  values  of  (3  and  higher  values  of  a.  As  a rule,  deconvolution  is  better  behaved 
at  lower  values  of  (3  than  it  is  when  (3  « 1.  A significant  discovery  is  that  an  image  blurred  with  a 
pair  ( Q'o,  A) ) can  often  be  successfully  deblurred  with  an  appropriate  pair  (o,/i),  where  er  > cvo  and 
(3  < f3 o . An  example  of  this  phenomenon  was  shown  in  Figure  2(D)  in  connection  with  the  blurred 
New  Orleans  cathedral  image.  An  effective  interactive  framework  for  performing  the  above  least 
squares  fitting  is  the  fit  command  in  DATAPLOT.2 1 This  is  a high-level  English-syntax  graphics 
and  analysis  software  package  developed  at,  the  National  Institute  of  Standards  and  Technology. 
This  software  tool  was  used  throughout  this  paper. 

The  following  version  of  the  APEX  method,  using  the  SECB  marching  backwards  m time  option 
(11),  has  been  found  useful  in  a variety  of  image  enhancement  problems  where  the  image  g{x,  y)  is 
such  that  In  | p*(£,  0)|  is  generally  globally  monotone  decreasing  and  convex,  as  shown  in  Figure 
4(A).  Choose  a value  of  .4  > 0 in  (22)  such  that  the  least  squares  fit  develops  a well-formed  cusp 
at  f = 0,  as  shown  in  Figure  4(B).  Using  the  returned  pair  {aj3)  in  the  SECB  method,  obtain  a 
sequence  v'(x,y,t)  of  partial  restorations  (11),  as  t decreases  from  i = 1,  as  illustrated  in  the  MR  I 
image  sequence  in  Figure  5.  With  a good  choice  of  A,  high  quality  restorations  will  be  found  at 
positive  values  of  t.  and  these  will  gradually  deteriorate  as  t — > 0.  Typically,  the  restoration  at  t — 0 
will  exhibit  undesirable  artifacts,  indicating  that  continuation  backwards  in  time  has  proceeded  too 
far  in  (9).  If  the  pair  (aj3)  produces  a high  quality  restoration  at  t = t\  > 0,  the  pair  (cq,/3), 
where  aq  = (1  — t\)a,  will  produce  the  same  quality  results  at,  t = 0.  In  general,  there  will  be  many 
values  of  A in  (22)  returning  pairs  (cv,  (3)  that  produce  good  reconstructions  at  some  t,Qp  > 0.  A 
large  number  of  distinct  pairs  ( a*,f3 *)  can  thus  be  found  that,  produce  useful,  but  distinct,  results 
at  t = 0. 

We  have  been  assuming  h(ft  rj)  to  be  a pure  Levy  otf  in  (15).  For  more  general  class  G otfs  (2), 
we  may  still  use  the  approximation  ln|</*(£,  0)|  ~ — aj£|2^  — A,  and  apply  the  same  technique 
to  extract  a suitable  pair  (a,  f3)  from  the  blurred  image.  Here,  the  returned  APEX  values  may 
be  considered  representative  values  for  the  cq,  ft  in  (2),  that  produce  a single  pure  Levy  otf 
aproximating  the  composite  otf. 
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7.  Application  to  SEM  images 

It  will  be  helpful  to  recall  some  basic  properties  of  APEX  processing  in  the  discussion  below.  Given 
a sharp  image  f(x,y),  convolution  of  that  image  with  any  class  G psf  to  form  a blurred  image 
g{x,  y),  is  mathematically  equivalent  to  a.  heat  conduction  process  in  which  bright  areas  in  f(x,y) 
correspond  to  hot  spots,  and  dark  areas  to  cold  spots.  As  time  progresses,  heat  conduction  acts 
so  as  to  diminish  temperature  differences.  As  a result,  bright  areas  in  f(x,y)  become  dimmer  in 
g(x,  y ),  while  dark  areas  in  f{x,y)  become  lighter  in  g{x,y).  This  causes  a smoothing  out  of  sharp 
edges,  a loss  of  structural  detail,  and  a decrease  in  contrast  in  g{x,y). 

APEX  deblurring  is  the  converse  process.  Given  a blurred  image  g(x,y),  deconvolution  of  that 
image  with  a class  G psf  is  equivalent  to  a reverse  heat  conduction  process.  Now,  some  light  areas 
in  g{x,  y)  become  brighter,  while  some  gray  areas  become  darker.  There  is  a sharpening  of  edges,  a 
gain  in  structural  detail,  and  a necessary  increase  in  contrast.  Inevitably,  there  is  also  an  increase 
in  noise.  By  performing  the  deconvolution  in  slow  motion , using  the  marching  backwards  in  time 
option  (11),  we  can  monitor  this  reverse  heat  flow,  and  terminate  the  process  at  some  time  f0  > 0 
before  brightness,  contrast,  or  noise,  become  excessive. 

Our  first  reconstruction  experiments  are  displayed  in  Figures  6 through  10.  In  each  of  Figures  6, 
7 and  8,  the  top  row  contains  the  original  SEM  images  that  were  used  as  input  data  into  the  APEX 
method.  The  bottom  row  contains  the  corresponding  APEX-processed  images.  The  middle  row 
in  each  of  these  Figures  was  synthesized  after  acquiring  and  viewing  the  bottom  row  images.  To 
minimize  the  effects  that  contrast  and  brightness  have  on  perception,  the  middle  row  images  were 
created  by  readjusting  contrast  and  brightness  in  the  original  top  row  images,  so  as  to  more  closely 
match  that  found  in  the  bottom  row  as  a result  of  APEX  processing.  Therefore,  comparing  the  top 
row  with  the  bottom  row  in  Figures  6-8  shows  the  full  effect  of  APEX  processing,  while  comparing 
the  middle  row  with  the  bottom  row  isolates  the  sharpening  aspect  of  APEX  processing.  As  might 
be  expected,  the  vivid  differences  between  these  three  rows,  which  are  immediately  apparent  on 
a modern  high-resolution  computer  screen,  have  become  muted  on  the  printed  page.  Accordingly, 
use  of  a magnifying  glass  may  be  helpful  in  parts  of  the  following  discussion.  In  Figures  9 and  10, 
selected  magnified  portions  of  the  contrast-enhanced  and  APEX-processed  images  in  Figures  6-8 
are  compared.  These  enlargements  provide  good  illustrations  of  the  detection  of  hue  structure  as  a 
result  of  APEX  sharpening. 

All  original  micrographs  were  input  as  8-bit  512  x 512  images,  although  smaller  sub  images 
are  displayed  in  some  cases.  Figures  6(A),  6(D)  and  7(A)  are  original  images  taken  by  John 
Small  (NIST),  on  a Hitachi  S-4500  held  emission  scanning  electron  microscope.  All  three  images 
are  micrographs  of  a complex  multi-form  crystalline  compound  of  mercury.  The  held  of  view  is 
10pm  in  Figure  6(A),  200pm  in  Figure  6(D),  and  20pm  in  Figure  7(A).  Figure  7(D)  is  an  image  of 
a 2 pm  diameter  hy  ash  particle  on  a Nuclepore  filter.  The  hlter  was  a backup  for  an  impactor  air 
sampler.  The  image  was  scanned  from  a Polaroid  print  taken  by  John  Small  (NIST)  in  the  1970's, 
on  a Cambridge  SEM1  at  the  University  of  Maryland.  Figure  8(A)  is  a micrograph  of  a dust  particle 
from  an  air  vent.  This  is  a complex  agglomerate  of  biological  and  mineral  particles.  Figure  8(D)  is 

* Certain  commercial  equipment  or  products,  including  hardware  and  software  components,  are  identified  in  this 
paper  to  adequately  describe  experimental  procedures.  Such  identification  does  not  imply  recommendation  or  en- 
dorsement by  the  National  Institute  of  Standards  and  Technology,  nor  does  it  mean  that  the  equipment  or  products 
so  identified  are  necessarily  the  best  available  for  the  purpose. 
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Figure  6:  Top  row  images  are  original  8-bit  512  x 512  SEM  micrographs,  used  as  input  data  in 
APEX  method.  Field  of  view  is  10 pm  in  image  (A)  and  200pm  in  image  (D).  APEX-processed 
images  are  in  bottom  row.  Middle  row  images  obtained  by  readjusting  contrast  in  top  row  images 
to  match  contrast  in  APEX-processed  bottom  row.  Comparing  bottom  row  with  top  row  shows 
full  effect  of  APEX  processing.  Comparing  bottom  row  with  middle  row  isolates  sharpening  aspect 
of  APEX  method.  Highlighted  areas  in  middle  row  indicate  regions  of  interest.  See  accompanying 
discussion  in  section  7. 
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Figure  7:  Top  row  images  are  original  8-bit  512  x 512  SEM  micrographs,  used  as  input  data  in 
APEX  method.  Field  of  view  is  20 p/m  in  image  (A)  and  2 pm  in  image  (D).  APEX-processed 
images  are  in  bottom  row.  Middle  row  images  obtained  bv  readjusting  contrast  in  top  row  images 
to  match  contrast  in  APEX-processed  bottom  row.  Comparing  bottom  row  with  top  row  shows 
full  effect  of  APEX  processing.  Comparing  bottom  row  with  middle  row  isolates  sharpening  aspect 
of  APEX  method.  Highlighted  areas  in  middle  row  indicate  regions  of  interest.  See  accompanying 
discussion  in  section  7. 
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Figure  8:  Top  row  images  are  250 firm  field  of  view  original  8-l)it.  512  x 512  SEM  micrographs,  used 
as  input  data  in  APEX  method.  APEX-processed  images  are  in  bottom  row.  Middle  row  images 
obtained  by  readjusting  contrast  in  top  row  images  to  match  contrast  in  APEX-processed  bottom 
row.  Comparing  bottom  row  with  top  row  shows  full  effect  of  APEX  processing.  Comparing  bottom 
row  with  middle  row  isolates  sharpening  aspect  of  APEX  method.  Highlighted  areas  in  middle  row 
indicate  regions  of  interest.  See  accompanying  discussion  in  section  7. 
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Figure  9:  Comparison  of  contrast-enhanced  and  APEX-processed  images  illustrates  detection  of 
small-scale  structure  through  image  sharpening.  (A)  Magnified  portion  of  image  8(B).  (B)  Corre- 
sponding portion  of  image  8(C).  (C)  Magnified  portion  of  image  7(B).  (D)  Corresponding  portion 
of  image  7(C). 
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Figure  10:  Comparison  of  contrast-enhanced  and  APEX-processed  images  illustrates  detection  of 
small-scale  structure  through  image  sharpening.  (A)  Magnified  portion  of  image  6(B).  (B)  Corre- 
sponding portion  of  image  6(C).  (C)  Magnified  portion  of  image  8(E).  (D)  Corresponding  portion 
of  image  8(F). 
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a micrograph  of  a.  brass  filing  particle.  Both  of  these  were  taken  on  a Hitachi  S-4500,  and  the  field 
of  view  is  250 //m  in  each  case. 

In  all  cases  in  Figures  6-8,  the  APEX  method  was  applied  to  the  original  top  row  images  on 
the  discrete  frequency  interval  |£|  < 256,  and  with  an  apex  value  of  A = 3.85.  This  produced  a 
well-defined  cusp  at  = 0 as  illustrated  in  Figure  4(B).  Different  pairs  (a,  fl)  were  detected  for  each 
image.  The  SECB  method  was  then  applied  with  the  detected  pairs  ( a,  (3 ),  and  with  s — 0.001 
and  K = 1.0.  Values  of  t0  > 0 were  chosen  in  each  case  to  select  the  ‘best,’  reconstruction.  This 
choice  of  t0  is  partly  subjective,  but  also  depends  in  part  on  the  particular  features  that  need  to  be 
resolved.  In  general,  images  that  are  less  sharpened  seem  more  pleasing  to  the  eye,  while  images 
that  are  more  sharpened  make  surface  detail  and  small  decorations  visible,  albeit  with  noticeable 
background  noise. 

In  Figure  6(A)  the  detected  psf  values  were  a = 0.6165,  / 3 — 0.1913,  and  and  a value  to  = 0.8 
was  used  to  produce  Figure  6(C).  Although  the  original  image  6(A)  appears  sharp  with  adequate 
contrast,  more  fine  surface  detail  on  the  central  particle  becomes  visible  in  the  contrast-enhanced 
image  6(B).  However,  in  enhancing  the  surface  detail  on  the  central  particle  in  image  6(B),  other 
parts  of  the  image  suffer.  For  example,  detail  near  the  bright  edges  in  the  lumpy  objects  in  the 
upper  right,  (see  highlighted  area  in  Figure  6(B)),  as  well  as  detail  in  the  lumpy  objects  in  the  lower 
left,  has  been  washed  out.  On  the  other  hand,  the  APEX-processed  image  6(C)  shows  even  more 
fine  detail  in  the  central  particle,  while  also  showing  more  fine  surface  structure  on  the  lumps  in 
upper  right  and  lower  left  corners. 

In  Figure  6(D)  the  detected  psf  values  were  a = 0.5346,  (3  — 0.2097,  and  a value  t0  = 0.88 
was  used  to  produce  Figure  6(F).  Since  image  6(D)  already  has  high  contrast,  there  is  not  much 
difference  between  it  and  image  6(E),  the  contrast-enhanced  version  of  6(D).  However,  the  APEX 
image  6(F)  has  even  more  contrast,  which  helps  bring  out  fine  surface  detail  barely  visible  in  the 
other  two  images.  The  APEX  image  6(F)  also  has  sharper  and  brighter  edges,  making  the  three- 
dimensional  form  form  of  this  complex  particle  easier  to  understand. 

In  Figure  7(A)  the  detected  values  were  cv  = 0.6915,  [3  = 0.1641,  and  t0  = 0.64  was  used  to 
produce  Figure  7(C).  This  particular  sample  has  very  complex  and  varied  morphology,  in  addition 
to  surface  dusting  or  decoration  of  fine  particles  almost  everywhere.  This  becomes  clearly  evident 
only  in  the  APEX  image  7(C).  Mere  contrast  enhancement  does  not  produce  as  much  detail  in  the 
highlighted  area  in  image  7(B),  as  is  visible  in  the  corresponding  area  in  7(C).  Moreover,  contrast 
enhancement  in  7(B)  also  tends  to  obscure  texture  in  the  brighter  areas,  such  as  in  the  lower  left 
corner.  However,  the  APEX  image  clearly  shows  the  texture  in  the  lower  left  corner  as  well  as 
in  other  bright  areas.  It  also  retains  the  three-dimensionality  of  the  particle  by  not  eliminating 
shading,  as  is  often  the  case  with  high-pass  filtering. 

In  Figure  7(D)  the  detected  psf  values  were  cv  = 0.9311,  f3  = 0.1441,  and  a value  = 0.4  was 
used  to  produce  Figure  7(F).  This  image  is  unlike  the  other  images,  in  that  it  was  scanned  from  an 
old  Polaroid  print  rather  than  scanned  digitally  on  the  microscope.  Imperfections  on  the  Polaroid 
print  are  detected  by  APEX  processing,  along  with  enhancing  the  texture  in  the  sample.  Some  of 
that  texture  may  be  due  to  the  print  rather  than  to  the  sample  itself.  Nevertheless,  this  example 
is  a useful  illustration  of  t he  APEX  method’s  ability  to  detect  fine  structure.  Presumably,  actual 
imperfections  or  small  defects  in  some  other  sample  might  have  been  detected  equally  well.  While 
the  scratches  near  the  center  are  visible  in  all  three  images,  the  scratch  near  the  top  right  corner 
(see  highlighted  area  in  image  7(E)),  is  clearly  discernible  only  in  the  APEX  image  7(F).  Further, 
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the  edges  in  the  APEX  image  are  sharper  or  less  washed  out  than  in  the  other  two  images.  This 
makes  the  image  have  more  depth;  the  structure  in  the  lower  left  quadrant  appears  closer  than  does 
the  rest  of  the  image. 

In  Figure  8(A)  the  detected  psf  values  were  a = 0.2981.  f3  = 0.2210,  and  a value  of  f0  = 0.44 
was  used  to  produce  Figure  8(C).  Contrast  enhancement  in  Figure  8(B)  makes  the  complex  form  of 
the  sample  easier  to  see,  while  bringing  out  hue  particles.  However,  as  in  the  previous  cases,  such 
enhancement  also  obscures  detail  in  the  brighter  areas,  as  in  the  highlighted  area  in  Figure  8(B), 
for  example.  The  APEX  image  8(C)  has  brighter  edges  than  the  original  8(A),  and  sharper  edges 
than  the  contrast  enhanced  8(B).  Moreover,  fine  detail  becomes  visible  both  in  the  medium  and 
bright  areas  of  the  image. 

In  Figure  8(D)  the  detected  psf  values  were  cv  = 0.7634,  j3  — 0.1827,  and  a value  t0  = 0.6  was 
used  to  produced  Figure  8(F).  The  contrast  enhanced  Figure  8(E)  is  easier  on  the  eyes,  but  does  not 
have  more  visible  detail  than  does  the  original  Figure  8(D).  The  APEX  image  8(F)  has  thinner  or 
less  washed  out  edges,  making  fine  detail  (which  in  this  image  is  mostly  in  the  edges)  much  easier  to 
see.  The  highlighted  area  in  Figure  8(E)  is  one  example  of  a structure  that  is  more  sharply  defined 
in  Figure  8(F). 

In  Figures  9 and  10.  selected  enlarged  portions  of  some  of  the  contrast-enhanced  and  APEX- 
processed  images  in  Figures  6-8  are  displayed  side  by  side.  Comparing  these  enlargements  empha- 
sizes some  of  the  points  made  above,  and  provides  a good  illustration  of  the  level  of  fine  structure 
that  may  be  revealed  as  a result  of  APEX  sharpening. 

8.  Etched  grass  image  and  quantitative  APEX  sharpness  analysis 

In  periodic  performance  testing  of  scanning  electron  microscopes,  sharpness  degradation  in  the 
micrograph  of  a suitable  test  object  is  often  used  as  an  indicator  of  the  need  for  maintenance.  The 
properties  of  an  ideal  test  object  for  this  purpose  are  discussed  in  Refs.  1 and  2.  These  properties 
include  geometric  requirements,  as  well  as  the  ability  to  yield  reasonably  noiseless  images  with  good 
contrast  at  high  magnification.  A silicon  wafer  with  an  etching  artifact  called  ‘grass’  was  found  to 
meet  these  criteria,  and  was  used  in  Ref.  2.  In  the  present  paper,  the  same  grass  sample,  together 
with  ‘SEM  Monitor’  software,2  provides  a useful  evaluation  of  the  APEX  method. 

The  images  in  the  top  row  in  Figure  11  are  original  8-bit  512  x 512  images  obtained  from  the 
Hitachi  S-4700  field  emission  scanning  electron  microscope.  The  held  of  view  is  200nm  for  all 
images.  Image  (A)  is  as  sharp  as  could  be  achieved  with  optimal  settings  of  the  focusing  (objective) 
and  stigma  control  (X  and  Y)  lenses.  Image  (D)  is  out  of  focus.  It  was  taken  with  an  objective  lens 
setting  somewhat  above  that  used  for  image  (A),  the  sharpest  image.  Image  (G)  is  astigmatic  and 
was  taken  with  the  5’  stigma  control  set  to  a lion-optimal  value.  Images  (C),  (F),  and  (I)  in  Figure 
11,  are  the  corresponding  APEX-processed  images.  Each  of  these  was  selected  from  a sequence  of 
increasingly  sharper  images,  as  illustrated  with  the  MRI  image  sequence  in  Figure  5.  As  in  Figures 
6,  7 and  8,  the  middle  row  images  in  Figure  11  were  created  by  readjusting  the  contrast  in  the  top 
row  images,  so  as  to  more  closely  match  that  found  in  the  bottom  row  images  as  a result  of  APEX 
processing. 

The  following  APEX  parameter  values  were  used  in  all  three  cases,  A = 3.85  on  |£|  < 256,  s = 
0.001,  I\  = 1.0,  and  f0  = 0.9.  However,  different  pairs  (tv,  (3)  were  detected  for  each  image.  For  the 
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Figure  11:  Top  row  images  are  200 nm  field  of  view  original  8-bit  512  x 512  SEM  micrographs 
of  ‘grass’  sample,  used  as  input  data  in  APEX  procedure.  Bottom  row  images  are  corresponding 
APEX  outputs.  Image  (A)  is  sharp,  image  (D)  is  out  of  focus,  and  image  (G)  is  astigmatic.  Middle 
row  images  obtained  by  adjusting  contrast  in  top  row  images  to  match  contrast  in  APEX-processed 
bottom  row.  ‘SEM  Monitor’  software  indicates  15%  increase  in  sharpness  after  APEX  processing. 
See  Table  1 and  accompanying  discussion  in  section  8. 
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sharp  image  (A),  a = 0.8508,  (3  = 0.1522;  for  the  out  of  focus  image  (D),  a = 1.1349,  (3  = 0.1305; 
for  the  astigmatic  image  (G),  a = 1.1129,  (3  = 0.1321.  Again,  comparing  the  top  row  with  the 
bottom  row  n Figure  11  shows  the  full  effect-  of  APEX  processing,  while  comparing  the  middle  row 
with  the  bottom  row  isolates  the  sharpening  aspect  of  the  APEX  method. 

‘SEM  Monitor'  is  a hardware  and  software  system  designed  to  provide  a quantitative  framework 
for  monitoring  performance  in  scanning  electron  microscopes.  Use  of  that  system  in  connection  with 
the  above  grass  sample  is  discussed  in  Ref.  2.  The  system  calculates  several  parameters,  including 
a quantitative  measure  of  image  sharpness.  Here,  we  use  that  system  to  measure  the  effect  of 
APEX  processing  on  each  of  the  three  original  micrographs  in  Figure  11.  The  results  displayed 
in  Table  1 indicate  sharpness  increases  on  the  order  of  15%  after  APEX  processing.  Interestingly, 
this  even  holds  true  for  the  sharpest  image  that  could  be  achieved,  image  11(A).  This  implies  that 
APEX  processing  may  be  used  to  extend  an  SEM’s  capability,  by  producing  sharper  imagery  than 
is  achievable  under  optimal  settings. 


TABLE  1 


Sharpness  improvement  after  APEX  processing  as  measured  by  'SEM  Monitor'. 


Origin al  sh arp n ess 

Detected  pair  (cv,  (3) 

APEX  sharpness 

Improvement. 

Image  11(A)  = 2.32 

a = 0.851,  (3  = 0.152 

Image  11(C)  = 2.G8 

15.5% 

Image  11(D)  = 2.19 

<y  = 1.135,  / 3 = 0.130 

Image  11(F)  = 2.51 

14.6% 

Image  11(G)  = 2.15 

a = 1.113,  / 3 = 0.132 

Image  11(1)  = 2.45 

14.0% 

9.  Concluding  remarks 

This  paper  has  demonstrated  the  use  of  a real-time  blind  deconvolution  technique  that  can  sharpen 
SEM  micrographs.  As  shown  in  section  7,  such  deconvolution  enables  detection  of  small-scale 
features  not  immediately  apparent  in  the  original  micrograph.  In  section  8,  APEX  processing  of 
ideal  test  sample  micrographs  produced  measured  increases  in  sharpness  on  the  order  of  15%.  While 
not  all  SEM  images  can  be  significantly  improved,  these  results  indicate  the  APEX  method  to  be 
a useful  tool  in  electron  microscopy.  Successful  applications  of  APEX  processing  in  several  other 
imaging  modalities,  unrelated  to  SEM,  have  previously  been  documented.4 

The  APEX  method  is  predicated  on  two  assumptions.  The  first  assumption  is  that  the  blurred 
image  g(x , y)  obeys  the  simple  convolution  equation  (5)  rather  than  a more  general  integral  equation. 
The  second  assumption  is  that  the  point  spread  function  h(x,y)  belongs  to  a restricted  class  of 
unimodal.  radially  symmetric,  probability  density  functions,  the  class  G defined  in  (2).  It  is  not 
immediately  obvious  that  the  APEX  method  can  be  usefully  applied  in  electron  microscopy. 


THE  APEX  METHOD  IN  ELECTRON  MICROSCOPE 


24 


The  range  of  (3  values  that  were  detected  and  used  in  Figures  6-11,  is  interesting  in  its  own 
right.  The  exponent  [3  expresses  the  degree  of  departure  from  more  commonly  occurring  Gaussian 
densities  where  [3  = 1.0.  Here,  0.13  < (3  < 0.22.  A similar  range  of  values  values  for  / 3 was  found 
in  Ref.  4.  As  noted  in  Figure  2,  there  are  several  Levy  pairs  (a,  (3)  that  can  produce  useful  recon- 
structions, and  higher  values  of  / 3 might  have  been  successfully  employed.  However,  experiments 
indicate  that  the  useful  [3  values  in  Figures  6-11  typically  lie  in  the  range  0 < (3  < 1/2.  Future  work 
will  explore  possible  links  between  such  (3  values  and  physical  processes  underlying  SEM  imaging. 
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